AI is not IT- and adopting artificial intelligence is almost nothing like adopting traditional software solutions. While software is deterministic, AI is probabilistic. The process of coaxing value from data with algorithms is a challenging and often time-consuming one. While non-technical AI project leaders and executives don't need to know how to clean data, write Python, or adjust for algorithmic drift – but they do have to understand the experimental process that subject-matter experts and data scientists go through to find value in data. Last week we covered the three phases of AI deployment, and this week we'll dive deeper in the seven steps of the data science lifecycle itself – and the aspects of the process that non-technical project leaders should understand.

Progressive is one of the largest auto insurers in the US. The company has been experimenting with AI since the middle of the 2010s, with customer-facing applications that update insurance premiums based on driving habits and answer questions in a chat window. In this article, we discuss both of these AI use-cases. Emerj's AI Opportunity Landscape research in insurance shows that Progressive's Snapshot program follows a trend in which auto insurers use predictive analytics applications to determine how risky a customer or insurance applicant is: Approximately 21.6% of AI products in insurance are applications of this type. In contrast, although several of the largest insurance companies in the US have experimented with chatbots, the insurance industry hasn't prioritized chatbots the way the retail industry has, as chatbots make up only 8% of the AI products in insurance.

This approach to predictive analytics applications can be illustrated by an example. Let's consider an e-commerce company that wants to boost its profits by growing sales to existing customers. The objectives might be to increase both the number of items bought by individual customers and the average amount they spend overall in purchase transactions. A typical strategy to accomplish those goals involves using a recommendation engine to try to influence customers to add items to their online cart as they shop. There are a variety of different analytics methods that the online retailer can incorporate into its recommendation engine to assign similar customers to groups so the engine can suggest products that they might be inclined to buy.

To stay competitive in a digital economy, businesses increasingly need to move beyond simple reporting and descriptive analytics to a more predictive approach that puts artificial intelligence (AI) strategies to work to engage with customers in new ways. So how can you find a practical way to start applying AI in your business? One path forward follows three steps: leverage predictive models to improve how you engage with customers, put machine learning to work to improve those models, and then validate your models. In this post, I will focus on the validation of predictive models First let me provide a quick overview of predictive analytics and machine learning, and explain why validation is important when you apply these approaches. Predictive analytics is about using algorithms to predict the result of a measurement that you can't make, based on measurements that you can make.